{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'\\n    @Author: King\\n    @Date: 2019.06.25\\n    @Purpose: Graph Convolutional Network\\n    @Introduction:   This is a gentle introduction of using DGL to implement \\n                    Graph Convolutional Networks (Kipf & Welling et al., \\n                    Semi-Supervised Classification with Graph Convolutional Networks). \\n                    We build upon the earlier tutorial on DGLGraph and demonstrate how DGL \\n                    combines graph with deep neural network and learn structural representations.\\n    @Datasets: \\n    @Link : \\n    @Reference : https://docs.dgl.ai/tutorials/models/1_gnn/1_gcn.html\\n'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'''\n",
    "    @Author: King\n",
    "    @Date: 2019.06.25\n",
    "    @Purpose: Graph Convolutional Network\n",
    "    @Introduction:   This is a gentle introduction of using DGL to implement \n",
    "                    Graph Convolutional Networks (Kipf & Welling et al., \n",
    "                    Semi-Supervised Classification with Graph Convolutional Networks). \n",
    "                    We build upon the earlier tutorial on DGLGraph and demonstrate how DGL \n",
    "                    combines graph with deep neural network and learn structural representations.\n",
    "    @Datasets: \n",
    "    @Link : \n",
    "    @Reference : https://docs.dgl.ai/tutorials/models/1_gnn/1_gcn.html\n",
    "'''"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Understand Graph Attention Network\n",
    "\n",
    "From Graph Convolutional Network (GCN), we learned that combining local graph structure and node-level features yields good performance on node classification task. However, the way GCN aggregates is structure-dependent, which may hurt its generalizability.\n",
    "从图卷积网络（GCN），我们了解到结合局部图结构和节点级特征可以在节点分类任务上产生良好的性能。但是，GCN聚合的方式依赖于结构，这可能会损害其普遍性。\n",
    "\n",
    "\n",
    "One workaround is to simply average over all neighbor node features as in GraphSAGE. Graph Attention Network proposes an alternative way by weighting neighbor features with feature dependent and structure free normalization, in the style of attention\n",
    "一种解决方法是简单地平均所有邻居节点功能，如GraphSAGE中所示。图注意网络通过以注意的方式加权具有特征相关和结构自由规范化的邻居特征来提出另一种方法\n",
    "\n",
    "\n",
    "The goal of this tutorial:\n",
    "\n",
    "- Explain what is Graph Attention Network.\n",
    "- Demonstrate how it can be implemented in DGL.\n",
    "- Understand the attentions learnt.\n",
    "- Introduce to inductive learning.\n",
    "\n",
    "\n",
    "\n",
    "## Introducing Attention to GCN\n",
    "\n",
    "\n",
    "The key difference between GAT and GCN is how the information from the one-hop neighborhood is aggregated.\n",
    "GAT和GCN之间的关键区别在于如何聚合来自一跳邻域的信息。\n",
    "\n",
    "For GCN, a graph convolution operation produces the normalized sum of the node features of neighbors:\n",
    "\n",
    "$$\n",
    "h_{i}^{(l+1)}=\\sigma\\left(\\sum_{j \\in \\mathcal{N}(i)} \\frac{1}{c_{i j}} W^{(l)} h_{j}^{(l)}\\right)\n",
    "$$\n",
    "\n",
    "\n",
    "GAT introduces the attention mechanism as a substitute for the statically normalized convolution operation. (GAT引入了注意机制作为静态归一化卷积运算的替代。) Below are the equations to compute the node embedding $h_{i}^{(l+1)}$ of layer l+1 from the embeddings of layer l:\n",
    "\n",
    "![](img/GAT_1.png)\n",
    "\n",
    "\n",
    "![](img/GAT_2.png)\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
